Lab 6 Worksheet

Getting started with Qualtrics & Gorilla

Starting Developing Your Studies

By the end of the session, you will have:

  • Worked together in your groups to determine next steps in your studies; specifically, materials that are shared, and those that are individual to you. This will allow you to divide and conquer!
  • Registered for a Goldsmiths Qualtrics account.
  • Opened the ‘Mini-Dissertation Qualtrics Template’ in your account.
  • Started editing the template according to your needs.
  • Considered randomizing blocks in your survey to avoid order effects.
  • Set up scoring to save you time computing scores.
  • If your study requires complex stimulus presentation or reaction times, you’ll want to use Gorilla. A brief overview of valuable resources is available.

Work with your group to identify which elements of your study (measures, questionnaires, tasks) are shared and which are individual. This will allow you to start considering how to combine your energies to best effect in developing materials.

Most, if not all, of you, will use Qualtrics. There are some useful tools and tips to help you get up and running. Import the Mini-Dissertation Template and get to work building your study. For more complex tasks, Gorilla will be what you need.

Gordon’s Top Tip: Time spent preparing your materials really pays off – in terms of time, data quality, and confidence!

Working Out How to Start Developing Your Study

  1. Moving towards your individual research question: In your groups, and with your Critical Proposals in mind, discuss your projects. Do you all still agree on your original research question or how you thought you’d explore it? Have your ideas developed in any way? List your preferred IVs, DVs, and methodologies and compare notes. Look out for similarities and differences.

    It is normal that your approaches or interests may diverge a little as a result of your critical evaluations. This is great. If not, that’s fine too. The design of your individual Mini-Dissertation needs to be slightly different from your collaborators, so you can pursue your interests if they differ somewhat.

    Exploring different personality traits is a nice way to find distinctive IVs. If you are presenting stimulus materials, you could present different types, classes, or categories. An exploration of memory may include familiar or unfamiliar words, short or long words, or focus on another type of stimulus, such as numbers or shapes, different presentation durations, or recall delays. Or you could use different DVs, such as accuracy/error rate, or reaction time, or confidence; all interesting. If you are looking at life-outcomes as a DV (as many of you are), these can vary by domain (e.g. work or home-life) or how you quantify them; how do you quantify success or happiness or health? It may initially seem quite tricky to see how this might work, but your lab tutor is on hand to help!

  2. Sharing the workload: Following on from the question above, is there a ‘central’ task, measure, or stimulus-set in your project that potentially requires more effort to find or build? Does some aspect of your project involve the selection or creation of stimulus material?

    This might be a good thing to develop as a group, allowing you to make it as effective as possible. This is likely the core of your methods section and worth making fabulous. You can then duplicate this component and break into smaller groups to tailor it and collect the data if you need to or want to. But by working together as much as possible, you can refine this component and make it as accurate and impressive as possible with relative ease.

Opening a Goldsmiths Qualtrics Account

  1. Please go to this link: Goldsmiths Qualtrics

  2. Use your college email address and password. To get the full functionality, you MUST use your college login.

  3. If you accidentally set up a trial account, or use a different email, it can take 3 weeks to change. Follow these instructions on the VLE: VLE Instructions

  4. You will know if you have been successful if, when you log in, the URL starts with goldpsych.eu.qualtrics.com.

Importing the Mini-Dissertation Template

  1. Download the Mini-Dissertation Qualtrics Template HERE or from the VLE page.

  2. It will be a .qsf file, native to Qualtrics, and will only open in Qualtrics.

  3. To use the template, on the Homepage, click on ‘Create a new project’, select ‘Survey’, and ‘Get started’.

  4. Give your project a preliminary name (this can be changed later) and choose ‘Import a QSF file’ from the ‘How you want to start’ dropdown.

  5. Find the .qsf file you downloaded earlier using the ‘Choose file’ finder and then click ‘Create project’.

  6. You will likely be offered the option to ‘Take a tour’ of the Survey Builder. It takes two minutes and shows you how to change settings and to edit questions.

The Template

Please notice that, over and above the survey questions, the template includes four important elements; 1) an Information Sheet outlining the study, 2) Informed Consent, 3) GDPR, and 4) a Debrief delivered at the end of the survey.

You are required to submit copies of the information you supply in these sections in word doc format with your Ethics Application for storage on file. Simply copy the content in the relevant sections of your Qualtrics survey into 4 separate word documents for submission.

NB: If you run an experiment in person (this year OR next), you will be required to give each participant a hard copy of all of these documents and retain a signed Consent Form for your records. The process of developing these materials is valuable preparation - please be involved.

Once you’ve created your survey, you’re ready to start building. This page outlines how to add, delete, copy, and edit multiple questions in the Survey tab to build your required functionality. The Qualtrics help is extremely useful and gives extensive, easily-navigable help.

For example, you will want to add new questions into the body of the survey, corresponding with demographic information or a psychometric measure or questionnaire you have found.

New Questions in your Survey

This page is very useful to help select the best question type. There are many types, but you will find Matrix Tables, Multiple Choice, and Sliders particularly useful. You can present images, media and text by using the Text/Graphic options and Rich Text Editor.

Try to make the questions as easy to navigate as possible. Remember to use page breaks to make the flow more enjoyable.

You can use a Rich Content Editor to include more elaborate instructions and adding media or graphics. You can also use hyperlinks to connect to content on the web outside of your questionnaire. You might want to include a downloadable file, such as a debrief form or pdf of some information you think a participant may want to retain. This can be managed via the Rich Content Editor. Making a participant feel like they have learned something can be a powerful reward. Please be careful not to break copyright by sharing materials you haven’t produced yourself. If you propose to include such materials, this should be included in your Ethics Application.

Important safety and ethics considerations

Please only use your Goldsmiths Email address when running research and if you give your own details ALWAYS supply those of your Lab Tutor or the Module Coordinator. This affords you some protection and allows us to help deal with any questions.

It is important that you clearly outline recruitment strategies in your ethic application (i.e. how you propose to recruit and where you share your task) and you must ONLY share an approved task.

Block Options and randomisation

Sets of questions (or entire questionnaires) can be set up as ‘blocks’ within your survey. If you wish to randomize these blocks, you can, meaning that you can randomly shuffle the sequence of presentation. This is an important feature that you may wish to consider using. Obviously, the information and informed consent will need to remain at the start and the debrief needs to always be the last block presented.

In the ‘survey flow’ area, you will see that you can view the blocks in the survey and add a randomizer, that randomly delivers a selection of blocks. Block 1 and 2 are identical in the template, but will be presented in random order for each new participant.

Recoding and Scoring (Massive time savers!)

Recode Values

Once you have set up your survey, it is always a good idea to run a test and export the data to make sure it is in the format that you require. If you notice that the data are not in the format you would most easily use, it may be that you could consider ‘recoding’ the values to make it easier to use.

Scoring questionnaires

Scoring can save you LOTS of time, by automatically scoring your questionnaires or subscales of your questionnaires. Consider using this if your measure generates a total score or you are planning on calculating an average etc. It is perfectly easy to calculate these in excel, but it can be done automatically with a little care and preparation.

Previewing your survey

If you want to see how a question (or entire survey) looks, use the preview functionality. It will allow you to see what the participant will see – both on a computer and on a mobile device.

Data Export

It is highly recommended that you test how the data will be exported from your survey, so that you can perfect how you label your questions and how you recode or score your variables. To run a quick ‘data test’ do the following.

Click Publish and label the versions ‘Data Test’ and click ‘Publish’ again.

Under ‘Distributions’ select ‘Anonymous link’ (which is how you will share your study with participants usually. You can of course produce a QR code or use Bit.ly to shorten the link to make it more attractive. Think about the best way to recruit for your study.

Copy the link to your browser and run through the study.

Export Data from this menu to a range of useful formats.

In Term 2 we will be looking at the data and how to clean it. The most important thing I would advise you to do is to name your blocks and questions clearly, so that this is reflected in the dataset that you download. If you don’t know which question is which, you could come unstuck.

Collaborate your survey with colleagues

If you wish to share your survey with your group and work on it together, you can ‘Collaborate’ the project. It’s like using a shared document. You can monitor versions in case you make a mistake, but remember to ‘Publish’ if you want to over-write any changes you make! You can find this in the ‘Tools’ menu and you will need to use your colleagues’ Goldsmiths email addresses AND they will need to have a Goldsmiths Qualtrics account too.

For More Information the Qualtrics website includes comprehensive and easy-to-follow help documentation as mentioned above. Tutorials, webinars and detailed directions are available via Qualtrics’ extensive online resource, Qualtrics Support. www.qualtrics.com/support

For fast, individualized support over chat, email or a scheduled phone call, please consider using the Support Center. We pay for the services here, but they are based in the US: www.qualtrics.com/support-center

For fast, individualized support over chat, email or a scheduled phone call, please consider using the Support Center. We pay for the services here, but they are based in the US: www.qualtrics.com/support-center

Gorilla.sc experimental platform

Some of you may be keen to program a task, such as a Stroop Task or a Go/No-Go task. For anything requiring complex presentation of stimuli or data recording accuracies or reaction times, Gorilla would be the most suitable platform to employ. Review some of the tasks they have in their ‘Samples’ collection ready for cloning! https://app.gorilla.sc/support/samples

Getting your Gorilla account if you are a new user to the software please follow the below guidance:

Please go the Gorilla webpage (https://gorilla.sc/login), if you don’t already have an account create a new one, at the bottom of STEP 3 - finish, you should find a box called + My institution already has a subscription. Please click on this box (please see an image below for ease) and enter the enrolment code: PsychGold